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This paper presents an experimental framework for 
computing opinion polarity summary from free-form 
unstructured textual reviews of a movie. The system works 
on movie reviews obtained from the World Wide Web and 
computes a sentiment polarity summary in an automated 
manner. The experimental design is based on use of the 
SentiWordNet Lexicon. Two labeled movie review datasets 
are used to evaluate the design. The standard performance 
evaluation metrics of accuracy, f-measure and entropy are 
calculated to demonstrate the suitability of the proposed 
framework. 
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INTRODUCTION 

Sentiment analysis is language processing task that uses an algorithmic formulation 
to identify opinionated content and categorize it as having “positive”, “negative” or “neutral” 
polarity. It has been formally defined as an approach that works on a 

quintuple"' T[ >. w h ere? °e is the target object, is a feature of the object 

KlJL is the sentiment polarity (+ve, -ve or neutral) of opinion of holder K on I L feature of 

object 1 at time 1 , and 1E is the time when the opinion is expressed (Liu, 2009). 

Sentiment analysis is now a very useful task across a wide variety of domains. 
Whether it is commercial exploitation by organizations for identifying opinions about 
products/services from various customers, or identifying the election prospect of political 
candidates; sentiment analysis finds its applications. The user created information is very 
useful for companies which try to know the feedback about their products or services. This 
feedback helps them in taking informed decisions. However, the large number of reviews 
becomes information overload. Sentiment analysis fills this gap by producing a sentiment 
profile computed from a large number of user reviews about a product or service. A user is 
not required to read all the reviews of a product/service but s/he can have access to an 
overall sentiment profile which tells how many reviews are positive and how many are 
negative. 
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This paper presents a SentiWordNet based approach for sentiment classification of 
movie reviews. The document-level sentiment classification formulation designed utilizes 
diverse linguistic features ranging from adjectives to verbs. Two schemes, AAC (Adverb 
Adjective Combination) and AAAVC (Adverb Adjective Adverb Verb Combination) are 
explored for sentiment analysis task. The rest of the paper is organized as follows. Section 
2 describes the task and method of Sentiment Analysis. Section 3 presents the dataset used 
and implementation. Section 4 describes the results obtained and the paper concludes in 
section 5. 

SENTIMENT ANALYSIS 

There are mainly three type of techniques for sentiment classification of subjective 
texts: (a) using a machine learning based text classifier such as Naive Bayes, SVM or KNN; 
(b) using semantic orientation schemes of extracting relevant n-grams of the text and then 
labeling them either as positive or negative and consequently as the document; and (c) 
using the SentiWordNet based publicly available lexicon dictionary that provides positive, 
negative and neutral scores for the words. 

2.1 Naive Bayes Implementation 

It is a supervised probabilistic machine learning classifier. The sentiment analysis 
problem can be visualized as 2-class text classification problem that is classifying text 
documents in two polarity classes, namely “positive” polarity and “negative” polarity. In 
Naive Bayes classifier, the probability of document 1 being in class 1 is computed as. 

F(c|d) k F(c) n p ( w k 10 

where, the term aP| C - 1 refer to prior probability of a document occurring in class c and 
corresponds to the majority class. The expression ' M?k is the conditional probability of a 
term u ' k occurring in a document of class c . The term lP ^ ?k I c -‘ is interpreted to be the 
measure of how much evidence the term ’^contributes that c is correct class. The main idea 
in this classification is to classify the document based on statistical pattern of occurrence of 
terms. The goal in text classification using Naive Bayes is to determine the best class for a 
document. The best class in Naive Bayes classification is the maximum a-posteriori (MAP) 
class, computed as: 

c nuzp = arg m^cF(dli) = arg rn ax F(c) n F(w ft k) 

ISltSTId 

The probability values are computed from the training set. The multiplication of 
many conditional probability terms can be reduced by adding logarithms of probabilities. 
Therefore, above equation can be modified as: 

= arg m^[lo g F(£:) + F(w t |c}] 

1SKSfii 

In equation above, each ■’ c term refer to the weight which specify how good an 
indicator the term Wk is for class c , and in similar way the prior lo E^( c 3 indicates the 
relative frequency of class L (Manning et al., 2008). 
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2.2 Sentiwordnet Approaches 

Two different methods based on of SentiWordNet lexicon are implemented here. 
These methods vary in term of their feature selection, weight assignment and aggregation 
schemes. In the first implementation, every document (that was tagged with POS tags) is 
parsed to extract all terms with POS tag “adjectives”. Based on reported results in (Chesley, 
2005), (Karamibekr & Ghorbani, 2012), it is known that adverbs and verbs play an 
important role in sentiment analysis. Therefore, this paper tried to use “adverbs” and 
“adjectives” as features in implementations. In the second version, “adverb+adjective” and 
“adverb+verb” combines are used as features. The basic steps and information flow in the 
SentiWordNet-based methods is shown in Figure 1. 


Figure 1: Lexicon Based Approach General Architecture 



For each extracted term, SentiWordNet lexicon is scanned and the two scores 
(positive polarity and negative polarity strengths weighted over a number of synsets) are 
obtained. An expanded scan is performed by including the synonyms (obtained from 
WordNet) of the term, if the original term does not occur in the SentiWordNet lexicon. 
Hereafter, the two versions implemented are referred to as SWN (AAC) and SWN (AAAVC). 
Since “adverbs” modify the scores of succeeding terms, it needs to be decided as to what 
proportion the sentiment score of an “adjective” or a “verb” should be modified by the 
preceding “adverb”. The modifying weight (scaling factor) of adverb score is taken as 0.35 
based on the conclusions reported in (Karamibekr & Ghorbani, 2012). The other main issue 
that remains to be addressed is how should the sentiment scores of extracted 
“adverb+adjective” and “adverb+verb” combines, in a sentence of the document, should be 
aggregated. If sentiment polarity score total of an “adverb+adjective” combine is “x” and 
“adverb+verb” combine is “y”; then the net sentiment score of these two taken together will 
be x ~ 0,3;y , if the weightage factor for “adverb+verb” combine is 30 percent. The indicative 
pseudo-code for the SWN (AAC) and SWN (AAAVC) methods are given below, in Algorithm 
1 and 2, respectively. A careful look at the steps will help to understand the manner and 
rationale of the expressions. A more detailed discussion on SentiWordNet based 
implementations are reported in (Singh et al, 2013a, 2013b, 2013c). All implementations 
have been done in JAVA with NetBeans IDE. 
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Figure 2: Pseudocode of SWN (AAC) Algorithm 

Algorithm 1 Algorithm SWN (AAC) 

1: For each sentence, extract adv+adj combines. 

2: For each extracted adv+adj combine do: 

3: if score(adj ) = 0 then 
A: ignore it. 

5: end if 

6: if score ( ad!i'J > 0 then 
7: if sewrefadj) > 0 then 

3: fsAAc ( <idv , adj ') = mm(l f score [adj) 1 sf * score ( adv)) 

9: end if 

10: if score (ad?) < 0 then 

11: f SAAcfadVi adj) = mm(l T scorc(adj) — sf * score(aefer)} 

12: end if 

13: end if 

lA: if saarE(adv) < 0 then 
13: if score> 0 then 

16:: fsAAcladv. adj) = m£tr(— 1 > score (ad?) + sf * seore(adv)) 

17: end if 

IS: if score[adj) c 0 then 

10: / s AAC (adi' i adj ) = max (—1 , score ( adj) — sf * score( adv) ) 

20:: end if 

2l : end if 

22: Add the pastlve and negative scores to respective pools. 


Figure 3: Pseudocode of SWN (AAC) Algorithm 

Algorithm 2 Algorithm SWN (AAAVC) 

1: For each sentence, extract adv+adj and ad v+verb combines. 

2: For each extracted adv+adj combine do: 

3: if score (ad/) = 0 then 
1: ignore It. 

5: end if 

6: if score(adi') > 0 then 
7: if score(iutj) > 0 then 

S: /(rah 1 , -aefir) = mm(l, score (adj) + sf * score(adv)} 

D: end if 

10: if score(adj') < 0 then 

ll: f(adv T adi) = min{ 1 , score (adj ) - sf * score (adu)) 

12: end if 

13: end if 

id: if score (adv) < 0 then 
15: if score f ad? ) > 0 then 

16: f (adv,adj) = mar(- 1, score(adj) + sf * score(adv )) 

17: end if 

IS: if score (adj) < 0 then 

l?: f (adv, adj) = mar(- 1, scare{arfj) — sf * seorc(aiftf) ) 

20: end if 

21: end if 

22: For each extracted adv+verb combine do: 

23: if scone(uerb) = 0 then 
24: ignore it. 

25: end if 

26: if score(ndu) > 0 then 
27: if score (verft) > 0 then 

2S: f(adv, t'crb) = min(l, &core(verb) + sf * score{adv )) 

29: end if 

30: if scorefi'erfi) < 0 then 

3l: f (adv, verb) — min( 1 „ score(uerfr) - sf * score (adv )) 

32: end if 

33: end if 

34: if score(adi;) < 0 then 
35: if score (cert) > 0 then 

36: ffadv, verfr) = mar(-l, scar £ (verb) + sf * score(adu)) 

37: end if 

3S: if score (vert) < 0 then 

39: f (adv, verb) = 7nar(— 1, scoT’e(irer&) — sf * seore(fldu)) 

40: end if 

41: end if 

42: Add the pastive and negative scores to respective pools. 
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DATASET 

The experimental frameworks are implemented and run on two datasets collected. 
The datasets include two review sets of 800 reviews for Hindi and 800 for English movies. 
There are 10 reviews each for 80 Bollywood and 80 Hollywood movies from the IMDB, a 
popular movie review website. Table 1 describes the detail of datasets. 

Table 1: Details of Dataset Used 


Dataset 

No. of Reviews 

Avg. Length (in Words) 

Bollwood Movies 

800 

323 

Hollywood Movies 

800 

300 


RESULT 

The document-level sentiment analysis results on two datasets using two different 
implementations (SWN (AAC) and SWN (AAAVC)). The table 2 presents the Accuracy F- 
Measure and Entropy values with two implementations. The result shows that SWN 
(AAAVC) achieve better accuracy than SWN (AAC). The table 3 presents category wise 
accuracy percentage of movie reviews labeled as ‘positive’ or ‘negative’ by different methods. 
The table 4 presents the total number of ‘positive’ or ‘negative’ assigned by these two 
methods. The figure 4 presents results summary of SWN implementations. Experimental 
results are also obtained for Naive Bayes implementation. The table 5 and figure 5 present 
results of Naive Bayes implementation for the two datasets. Results with varying training 
data are computed and it can be clearly seen that higher training data results in higher 
accuracy. 

Table 2: Performance Result on three Different Dataset 


Methods 

Datasets 


Datasetl 

Dataset2 

SWN(AAC) 

Accuracy 

76.0% 

74.75% 

F-Measure 

0.6943 

0.6895 

Entropy 

0.2174 

0.2393 

SWN(AAAVC) 

Accuracy 

76.50% 

75.88% 

F-Measure 

0.6994 

0.6894 

Entropy 

0.2032 

0.2333 


Table 3: Category wise Accuracy Percentage Assigned by Two Approaches 


Methods 

Datasets 


Datasetl 

Dataset2 

SWN(AAC) 

Positive 

91.11% 

81.94% 

Negative 

41.63% 

56.25% 

SWN(AAAVC) 

Positive 

94.36% 

85.76% 

Negative 

37.95% 

50.44% 
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Table 4: Total Number of Positive and Negative Labels Assigned by Two Approaches 


Methods 

Datasets 


Datasetl 

Dataset2 

SWN(AAC) 

Positive 

506/555 

472/576 

Negative 

102/245 

126/224 

SWN(AAAVC) 

Positive 

519/550 

494/576 

Negative 

93/245 

113/224 


Figure 4: Accuracy Plot for Different Datasets 
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Table 5: Performance Result on Two Different Dataset 


Training Size 

Run 

D1 

D2 

Train: V 2 

Test: V 2 

Train Doc. (p : 
n) 

277:122 

288:112 

NB Acc. (Avg.) 

80.123 

76.62 

Train: 1/3 

Test: 2/3 

Train Doc.(p: n) 

185:81 

192:74 

NB Acc.(Avg.) 

75.744 

76.126 

Train: 2/3 

Test: 1/3 

Train Doc. (p : 
n) 

370:162 

384:148 

NB Acc. (Avg.) 

82.750 

78.871 


133 


Volume 02, Issue 03, Version II, Jul - Sep’ 2015 


































Generating Sentiment Profile of Movies from Unstructured Textual Reviews 


Figure 5: Accuracy Plot for Different Datasets 

84 



NB Acc. (Avg.) 

NB Acc. (Avg.) 

NB Acc.(Avg.) 

Tr: 2/3 Tt:l/3 

Tr: 1/2 Tt: 1/2 

Tr: 1/3 Tt:2/3 
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CONCLUSION 

The paper presents experimental evaluation of SentiWordNet based and Naive 
Bayes approaches for sentiment analysis. The two algorithmic implementations are 
implemented and evaluated on two movie datasets. Detailed results of accuracy of 
classification of both the approaches are presented. It is observed that accuracy levels 
obtained are reasonably good. These implementations can thus be used to generate a 
sentiment profile summary from reviews of a movie. The approaches can also be used with 
other review type data. 
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